Establishing the validity of a performance assessment in numeracy

Validity is the extent to which the inferences drawn from scores on a test or assessment can be justified empirically and theoretically. Establishing the validity of an assessment can be seen as an ongoing process of judgment, using different forms of evidence for substantiation. Traditionally validity has been defined within an evidential framework of three interrelated aspects: criterion, content and construct validity. Messick's (1989) integrated view of validity led to inferences drawn from the information gathered from the assessment being pivotal to establishing validity and extended the understanding of validity to the consequences of test or assessment use, and led to a consideration of construct validity as the over-arching evidential basis. Performance assessment, in which students provide some form of product or performance, typically takes place under less standardised conditions than traditional test forms. It has been suggested that validity standards should be different for this form of assessment, relating more directly to the specific performance. Students' observed performances on a complex performance task in a numeracy context, as well as on tests of mathematics and mathematical problem solving, were analysed using Rasch modelling techniques. Six criteria suggested by Messick for construct validity evidence, were applied to the data. Findings indicated that the performance assessment information validity was high, and provided data useful for multiple purposes. The implications of these findings for performance assessment are discussed.-2-Background In recent years, in Australian school systems there has been a shift away from traditional subject areas, such as mathematics and English, towards an emphasis on cross-curriculum competencies, such as numeracy and literacy. In some instances, this has been accompanied by changes in the nature of the enacted curriculum, for example, the New Basics project in Queensland (Education Queensland, 2000) and the Essential Learnings in Tasmania (Department of Education, 2002). These new conceptions of the curriculum are expected to provide opportunities for students to exercise higher order thinking skills, such as analysing, justifying and conjecturing. These shifts provide a challenge for assessment developers. In Australia, numeracy is described in terms of numerate behaviour: To be numerate is to use mathematics effectively to meet the general demands of life at home, in paid work, and for participation in community and civic life. In school education, numeracy is a fundamental component of learning, performance, discourse and critique across all areas of the curriculum. It involves the disposition to use, in context, a combination of: underpinning mathematical …